Real-Time Bidding by Reinforcement Learning in Display Advertising

The majority of online display ads are served through real-time bidding (RTB) --- each ad display impression is auctioned off in real-time when it is just being generated from a user visit. To place an ad automatically and optimally, it is critical for advertisers to devise a learning algorithm to cleverly bid an ad impression in real-time. Most previous works consider the bid decision as a static optimization problem of either treating the value of each impression independently or setting a bid price to each segment of ad volume. However, the bidding for a given ad campaign would repeatedly happen during its life span before the budget runs out. As such, each bid is strategically correlated by the constrained budget and the overall effectiveness of the campaign (e.g., the rewards from generated clicks), which is only observed after the campaign has completed. Thus, it is of great interest to devise an optimal bidding strategy sequentially so that the campaign budget can be dynamically allocated across all the available impressions on the basis of both the immediate and future rewards. In this paper, we formulate the bid decision process as a reinforcement learning problem, where the state space is represented by the auction information and the campaign's real-time parameters, while an action is the bid price to set. By modeling the state transition via auction competition, we build a Markov Decision Process framework for learning the optimal bidding policy to optimize the advertising performance in the dynamic real-time bidding environment. Furthermore, the scalability problem from the large real-world auction volume and campaign budget is well handled by state value approximation using neural networks.Comment: WSDM 201

A low cost soft mapper for turbo equalization with high order modulation

In an MMSE based turbo equalization system, a soft mapper calculates the symbol mean and variance based on extrinsic Log-Likelihood-Ratios (LLRs) information coming from a Soft-Input Soft-Output (SISO) decoder. In this paper, we investigate the complexity of this module, and in particular, we employ a 3-segment linear approximation approach to calculate the mean and variance of data symbols from LLRs. For FPGA and VLSI implementation, we propose novel piecewise linear functions which map LLR to the mean and variance directly without the use of any two-variable-input multipliers. Simulation results for 16-QAM and 64-QAM show that the no multiplier approach has close BER performance to the 3-segment linear approximation approach with multipliers. © 2012 IEEE

Design and manipulation of high-performance photovoltaic systems based on two-dimensional novel KAgSe/KAgX(X=S,Te) van der Waals heterojunctions

The realization of high-performance two-dimensional (2D) solar photovoltaic systems are both fundamentally intriguing and practically appealing to meet the fast-growing energy requirements. Since the limited application of single 2D crystals in photovoltaic, here we propose a family of 2D KAgSe/KAgX(X=S,Te) van der Waals heterostructures (vdWHs), which are constructed by combining two different KAgX layers through interlayer vdW interaction. After a systematic study and further regulatory research of these vdWHs based on the first-principles, numerous fascinating characteristics and physical mechanisms are obtained. Firstly, favorable potential applications of these vdWHs in photovoltaics are confirmed in virtue of their desirable optoelectronic properties, such as the robust stabilitis, moderate direct band gaps, type-II band alignments together with superior carrier mobilities, visible optical absorptions, power conversion efficiencys (PCEs) and photocurrents in their based photovoltaic devices. More importantly, when under varying vertical electric field Ez, a phase transition of band alignment from type-II to type-I of these vdWHs can be induced by the opposite band shifts between layers, which may enrich their applications in light-emitting diodes and lasers. Meanwhile, the PCE can be expanded up to 23%, and an obvious red-shift peak of the photocurrent in the visible light range are also obtained at different Ez. These fascinating tunable properties of KAgSe/KAgX vdWHs under varying Ez not only promote the improvement of their photoelectric performances, but the underlying mechanisms can also be applied to next experimental design and practical application of other 2D photovoltaic systems. Especially for the red-shift peak of the photocurrent, which is rarely found but highly desirable in practical visible photoelectric conversion.Comment: 11 pages, 7figure

A low cost soft mapper for turbo equalization with high order modulation

In an MMSE based turbo equalization system, a soft mapper calculates the symbol mean and variance based on extrinsic Log-Likelihood-Ratios (LLRs) information coming from a Soft-Input Soft-Output (SISO) decoder. In this paper, we investigate the complexity of this module, and in particular, we employ a 3-segment linear approximation approach to calculate the mean and variance of data symbols from LLRs. For FPGA and VLSI implementation, we propose novel piecewise linear functions which map LLR to the mean and variance directly without the use of any two-variable-input multipliers. Simulation results for 16-QAM and 64-QAM show that the no multiplier approach has close BER performance to the 3-segment linear approximation approach with multipliers. © 2012 IEEE

MAPK Signaling Pathway Alters Expression of Midgut ALP and ABCC Genes and Causes Resistance to \u3cem\u3eBacillus thuringiensis\u3c/em\u3e Cry1Ac Toxin in Diamondback Moth

Insecticidal crystal toxins derived from the soil bacterium Bacillus thuringiensis (Bt) are widely used as biopesticide sprays or expressed in transgenic crops to control insect pests. However, large-scale use of Bt has led to field-evolved resistance in several lepidopteran pests. Resistance to Bt Cry1Ac toxin in the diamondback moth, Plutella xylostella (L.), was previously mapped to a multigenic resistance locus (BtR-1). Here, we assembled the 3.15 Mb BtR-1 locus and found high-level resistance to Cry1Ac and Bt biopesticide in four independent P. xylostella strains were all associated with differential expression of a midgut membrane-bound alkaline phosphatase (ALP) outside this locus and a suite of ATP-binding cassette transporter subfamily C (ABCC) genes inside this locus. The interplay between these resistance genes is controlled by a previously uncharacterized trans-regulatory mechanism via the mitogen-activated protein kinase (MAPK) signaling pathway. Molecular, biochemical, and functional analyses have established ALP as a functional Cry1Ac receptor. Phenotypic association experiments revealed that the recessive Cry1Ac resistance was tightly linked to down-regulation of ALP, ABCC2 and ABCC3, whereas it was not linked to up-regulation of ABCC1. Silencing of ABCC2 and ABCC3 in susceptible larvae reduced their susceptibility to Cry1Ac but did not affect the expression of ALP, whereas suppression of MAP4K4, a constitutively transcriptionally-activated MAPK upstream gene within the BtR-1 locus, led to a transient recovery of gene expression thereby restoring the susceptibility in resistant larvae. These results highlight a crucial role for ALP and ABCC genes in field-evolved resistance to Cry1Ac and reveal a novel trans-regulatory signaling mechanism responsible for modulating the expression of these pivotal genes in P. xylostella

Effects of Remote Ischemic Preconditioning on Decreasing Troponin Release in Patients Not Taking Sulfonylureas After Cardiac Surgery – A Meta-Analysis

ABSTRACT Introduction: Remote ischemic preconditioning (RIPC) is a new noninvasive myocardial protection strategy that uses blood pressure cuf inflation to simulate transient non-fatal ischemia to protect the myocardium and reduce ischemia-reperfusion injury. Sulfonylureas may mask the effects of RIPC due to their cardioprotec-tive effect. This meta-analysis aimed to evaluate whether RIPC, in the absence of sulfonylureas, reduces troponin release in patients undergoing cardiac surgery. Methods: We conducted a meta-analysis of randomized controlled clinical trials to determine whether RIPC can reduce postoperative troponin release in cardiac surgery patients undergoing cardiopulmonary bypass without treatment with sulfonylureas. The data were normalized to equivalent units prior to the analysis. A random-effects model was used to provide more conservative estimate of the effects in the presence of known or unknown heterogeneity. Results: Six studies with a total of 570 participants were included. The analysis showed that troponin release was lower in the RIPC group than in the control group at six hours (test of standardized mean differences = 0, Z=3.64, P 60 minutes, RIPC reduced troponin release at six hours (Z=2.84, P=0.005), 24 hours (Z=2.64, P=0.008), and 48 hours (Z=2.87, P=0.004) postoperatively. Conclusion: In cardiac surgery patients who are not taking sulfonylureas, RIPC can reduce troponin release at six and 48 hours postoperatively; hence, RIPC may serve significant benefits in certain cardiac surgery patients

Roughage quality determines the production performance of post-weaned Hu sheep via altering ruminal fermentation, morphology, microbiota, and the global methylome landscape of the rumen wall

Roughage quality is a crucial factor influencing the growth performance and feeding cost of ruminants; however, a systematic investigation of the mechanisms underlying this is still lacking. In this study, we examined the growth performance, meat quality, ruminal fermentation parameters, rumen microbiome, and tissue methylomes of post-weaned Hu sheep fed low- or high-quality forage-based diets. Our results showed that sheep in the alfalfa hay (AG) and peanut vine (PG) groups exhibited better growth performance, slaughter performance, and meat quality than sheep in the wheat straw group (WG). The sheep in the AG possessed relatively higher contents of serum immunoglobins (IgA, IgG, and IgM) and lower contents of serum inflammation factors (TNF-α, IL-1β, IL-6, and IL-8) than those in the WG and the PG did. In addition, the levels of blood T lymphocytes (CD4+ and CD8+) and the CD4-to-CD8 ratio were significantly higher in the AG sheep than in the WG sheep and PG sheep. The concentration of ruminal NH3-N was highest in WG sheep, whereas the concentrations of individual and total short-chain fatty acids (SCFAs) were highest in the PG sheep. The length, width, and surface area of ruminal papillae were markedly different among the three groups, with the sheep in the PG being the most morphologically developed. The main ruminal microbes at the genus level include Prevotella 1, Rikenellaceae RC9 gut group, norank f F082, Ruminococcus 1, and Ruminococcus 2. The relative abundances of certain species are positively or negatively associated with fermentation parameters and growth index. For example, the fibrolytic bacteria Ruminococcaceae UGG-001 showed positive relationships with the concentration of SCFAs, except propionate. In addition, the relative abundances of fibrolytic bacteria (e.g., Ruminoccus 1) showed a negative relationship with starch-degrading bacteria (e.g., Prevotellaceae). The genome-wide DNA methylation analysis revealed that rumen tissues in the PG sheep and WG sheep occupied different global DNA methylomes. The genes with differentially methylated promoters were involved in known pathways (e.g., the FoxO signaling pathway) and the Gene Ontology (GO) terms (e.g., anatomical structure morphogenesis) pertaining to rumen development. Two candidate genes (ACADL and ENSOARG00020014533) with hyper- and hypo-methylated promoters were screened as potential regulators of rumen development. In conclusion, roughage quality determines sheep growth performance via directly influencing rumen fermentation and microbiome composition, and indirectly affecting rumen development at the epigenetic level

Biomechanical analysis and modeling of different vertebral growth patterns in adolescent idiopathic scoliosis and healthy subjects

<p>Abstract</p> <p>Background</p> <p>The etiology of AIS remains unclear, thus various hypotheses concerning its pathomechanism have been proposed. To date, biomechanical modeling has not been used to thoroughly study the influence of the abnormal growth profile (i.e., the growth rate of the vertebral body during the growth period) on the pathomechanism of curve progression in AIS. This study investigated the hypothesis that AIS progression is associated with the abnormal growth profiles of the anterior column of the spine.</p> <p>Methods</p> <p>A finite element model of the spinal column including growth dynamics was utilized. The initial geometric models were constructed from the bi-planar radiographs of a normal subject. Based on this model, five other geometric models were generated to emulate different coronal and sagittal curves. The detailed modeling integrated vertebral body growth plates and growth modulation spinal biomechanics. Ten years of spinal growth was simulated using AIS and normal growth profiles. Sequential measures of spinal alignments were compared.</p> <p>Results</p> <p>(1) Given the initial lateral deformity, the AIS growth profile induced a significant Cobb angle increase, which was roughly between three to five times larger compared to measures utilizing a normal growth profile. (2) Lateral deformities were absent in the models containing no initial coronal curvature. (3) The presence of a smaller kyphosis did not produce an increase lateral deformity on its own. (4) Significant reduction of the kyphosis was found in simulation results of AIS but not when using the growth profile of normal subjects.</p> <p>Conclusion</p> <p>Results from this analysis suggest that accelerated growth profiles may encourage supplementary scoliotic progression and, thus, may pose as a progressive risk factor.</p

Expression, Purification and Characterization of Arginase from Helicobacter pylori in Its Apo Form

Arginase, a manganese-dependent enzyme that widely distributed in almost all creatures, is a urea cycle enzyme that catalyzes the hydrolysis of L-arginine to generate L-ornithine and urea. Compared with the well-studied arginases from animals and yeast, only a few eubacterial arginases have been characterized, such as those from H. pylori and B. anthracis. However, these enzymes used for arginase activity assay were all expressed with LB medium, as low concentration of Mn2+ was detectable in the medium, protein obtained were partially Mn2+ bonded, which may affect the results of arginase activity assay. In the present study, H. pylori arginase (RocF) was expressed in a Mn2+ and Co2+ free minimal medium, the resulting protein was purified through affinity and gel filtration chromatography and the apo-form of RocF was confirmed by flame photometry analysis. Gel filtration indicates that the enzyme exists as monomer in solution, which was unique as compared with homologous enzymes. Arginase activity assay revealed that apo-RocF had an acidic pH optimum of 6.4 and exhibited metal preference of Co2+>Ni2+>Mn2+. We also confirmed that heat-activation and reducing regents have significant impact on arginase activity of RocF, and inhibits S-(2-boronoethyl)-L-Cysteine (BEC) and Nω-hydroxy-nor-Arginine (nor-NOHA) inhibit the activity of RocF in a dose-dependent manner